This document includes a partial source code listing of a preferred embodiment of the applicant""s invention. The code, listed in Appendix A and in the drawings, forms part of the disclosure of the specification. It is copyrighted. The copyright owner has no objection to facsimile reproduction by anyone of the patent document, including the copyrighted Appendix A and the drawings as they appear in the Patent and Trademark Office file or records, but otherwise reserves all rights.
This invention relates to the field of software plug-ins for multimedia file players and for other applications supporting ordered data flow files. More precisely, this invention defines a new field of software that allows plug-ins and content to be insulated from differences in underlying platforms and frameworks.
Multimedia player frameworks have become widespread. Later versions of the most popular Internet browsersxe2x80x94Microsoft""s Internet Explorer and Netscape""s Communicatorxe2x80x94include at least one player in the basic package. These are frameworks such as RealNetworks, Inc.""s RealPlayer(trademark) G2 family (collectively, the xe2x80x9cRealPlayer(trademark)xe2x80x9d); Microsoft Corporation""s Windows Media Technologies (NetShow(trademark)); Macromedia, Inc.""s Director(trademark); Apple Computer, Inc.""s QuickTime(trademark); and Sun Microsystems, Inc.""s Java(trademark) Media Framework (JMF). Most of these frameworks are extensible by means of plug-ins discussed below; some, notably JMF, are extensible by means of applications built on top of the frameworks. By being extensible we mean that a particular framework supports a set of interfaces exposed for interaction with additional software modules or components.
A framework is a set of interfaces, e.g., API""s and procedures, and a set of capabilities exposed for the purposes of extensibility. It need not operate on multimedia-type files, i.e., files ordered sequentially and supporting the concept of a time-line; frameworks generally need not operate on time- or otherwise-ordered data files. In the rest of this document, however, we will discuss predominantly frameworks operating on ordered data flow files. We will refer to such frameworks interchangeably as xe2x80x9cframeworks,xe2x80x9d xe2x80x9cextensible frameworks,xe2x80x9d xe2x80x9cextensible data flow-based frameworks,xe2x80x9d or by similar expressions; we do not imply any difference in the particular designation used, unless otherwise indicated.
Frameworks are extended by means of extensions, for example plug-ins. Plug-ins, also referred to as extension modules, are modules or components dynamically loaded at run-time. Extensible architecture and plug-ins have been used in commercial products, the RealPlayer(trademark) and Windows Media Technologies being two instances. They also have been described in various publications. See U.S. Pat. Nos. 5,838,906 and 5,870,559. (The specifications of these patents are incorporated by reference as if fully set forth herein.) A number of companies market plug-ins specifically for extending multimedia players.
One difficulty faced by plug-in developers is that a plug-in must be ported for each platform, i.e., for each hardware-operating system combination. Another difficulty lies in adapting a plug-in to various frameworks. Third difficulty, closely related to the first two, is that platform porting and framework adaptation require developers to have working knowledge of the various platforms and frameworks. One object of this invention is to provide a single interface that allows plug-in developers to build a single, platform independent version of a plug-in. Another object of this invention is to provide a uniform interface to the various frameworks so that a single plug-in can extend functionality of multiple frameworks. The third object of this invention is to facilitate development of content that can operate similarly, i.e., consistently, with different multimedia players (e.g., RealNetworks, Inc.""s RealPlayer(trademark) and Microsoft Corporation""s Windows Media Technologies (NetShow(trademark))), and platforms (TV, PC, set-top boxes). The fourth object of this invention invention is to accelerate the development of content and multimedia plug-ins by promoting reuse of existing software objects (e.g., objects written in Java(trademark) and JavaScript).
This invention is an abstraction layer providing a uniform interface between a framework and one or more plug-ins. In the preferred embodiment, the invention is a Synchronization Abstraction Layer (SAL) abstracting time-based frameworks into a common synchronization interface. The SAL synchronizes itself and other plug-ins to a time-line of the underlying frameworkxe2x80x94and it does that independently of the underlying framework. In other words, the plug-ins interact with the underlying framework through the SAL, rather than directly. Typically, the SAL is implemented on top of the synchronization of the Application Programming Interfaces (API""s) provided by the underlying frameworks. It has at least one point of coupling with the underlying framework: a method for providing the SAL with the current time.
The SAL includes a uniform cross-platform interface for the plug-ins. Preferably, the cross-platform interface has a functional core independent of the specific framework underlying the SAL.
Some of the features of this invention are listed below:
1. Secure e-commerce, full interactive experiences including text and voice chat, games, graphics, animations, and advertising can be integrated and synchronized with streaming multimedia such as video and audio;
2. It can be used to build a content framework that insulates content and plug-in developers from details and differences in hardware platforms, so that the same content or plug-in can run on desktop platforms (PC, Macintosh, Solaris, Linux), Televisions (GI, SA), or other kinds of devices (AOL-TV, screen phones);
3. It can be used to build a content framework that insulates content and plug-in developers from specifics of, or differences in, software platforms, so that the same content or plug-in can run on Microsoft NetShow(trademark), RealNetworks RealPlayer(trademark), Apple Quicktime(trademark), Sun Java(trademark) Media Framework or any other media system;
4. It can run without a network or on different types of networks, such as wireless, Intranets, or the Internet;
5. It can be used to synchronize arbitrary data types, including text, chat, graphics, video, audio, and active content like Java(trademark), JavaScript, or Flash;
6. The framework-independent layer can be implemented using different languages, including Java(trademark), HTML, XML, JavaScript, VBScript, or other languages;
7. When used for video synchronization, it can be used to synchronize arbitrary datatypes with different spatial objects in video, and with different temporal objects in video.